Skip to content

refactor(observability): migrate to juniper-observability register_or_reuse (Phase 2a)#95

Merged
pcalnon merged 2 commits intomainfrom
phase2/observability-helpers-migration
May 6, 2026
Merged

refactor(observability): migrate to juniper-observability register_or_reuse (Phase 2a)#95
pcalnon merged 2 commits intomainfrom
phase2/observability-helpers-migration

Conversation

@pcalnon
Copy link
Copy Markdown
Owner

@pcalnon pcalnon commented May 6, 2026

Summary

Phase 2a of the migration plan in juniper-ml/notes/observability/REGISTER_OR_REUSE_HELPER_DESIGN_2026-05-05.md.

Drops the inline _get_or_create helper added in PR #87 and calls the canonical juniper_observability.register_or_reuse shipped in juniper-observability 0.2.0.

What changes

File Change
juniper_data/api/observability.py _ensure_dataset_metrics swaps inline _get_or_create for register_or_reuse from juniper_observability. Drops the inline closure + REGISTRY._names_to_collectors lookup.
pyproject.toml juniper-observability>=0.1.1>=0.2.0

Drive-by: storage circular import (separate-but-blocking)

PR #92 added a top-level from juniper_data.api.observability import set_datasets_cached in juniper_data/storage/cached.py. That triggers a circular import via juniper_data.api.__init__app.pyjuniper_data.storage (still initialising), which broke pytest --collect-only on origin/main as of 2026-05-06 with 31 collection errors across every test that imports juniper_data.storage directly or transitively.

Surfaced during verification of this PR's main change. Fix is a deferred import inside CachedDatasetStore._emit_cached_count — the function is best-effort with an except Exception swallowing failures, so the deferred import slot in already-protected. Could be split into its own PR but the migration was wedged on it, so bundled here with explicit call-out.

Verification

Full juniper-data suite under JuniperData env: 950 passed (was 31 errors during collection on origin/main pre-fix).

Test plan

  • juniper-data full suite passes (950 / 950)
  • Adjacent test_observability.py (the SEC-16 / dataset-metrics regressions) still passes
  • No new metrics or labels — only the construction path changed
  • Dep pin bumped to >=0.2.0 so the helper resolves

Companion PRs

Part of Phase 2 — also coming: juniper-canopy and juniper-cascor migrations. juniper-cascor-client deliberately excluded from Phase 2 (would add pydantic + starlette as transitive deps to a leaf library; the lone PR #37 inline guard stays).

🤖 Generated with Claude Code

…_reuse (Phase 2a)

Phase 2a of the migration plan in
``juniper-ml/notes/observability/REGISTER_OR_REUSE_HELPER_DESIGN_2026-05-05.md``.
Drops the inline ``_get_or_create`` helper added in PR #87 and calls
the canonical ``juniper_observability.register_or_reuse`` shipped in
juniper-observability ``0.2.0``.

### What changes

- ``juniper_data/api/observability.py``: ``_ensure_dataset_metrics``
  now imports ``register_or_reuse`` from ``juniper_observability``
  (lazy, inside the ``if _dataset_metrics is None`` branch) and uses
  it for all four collectors (Counter ×2 + Histogram + Gauge). The
  inline ``_get_or_create`` closure and the manual
  ``REGISTRY._names_to_collectors`` lookup are gone.
- ``pyproject.toml``: ``juniper-observability>=0.1.1`` →
  ``juniper-observability>=0.2.0``. The new helper is the reason for
  the bump; existing 0.1.1 callers' behaviour is unchanged.

### Drive-by fix: storage circular import

PR #92 (``audit-fixup: wire juniper_data_datasets_cached gauge to
cache layer``) added ``from juniper_data.api.observability import
set_datasets_cached`` at the top of
``juniper_data/storage/cached.py``. That triggers the import chain:

    juniper_data.storage.__init__
      → cached.py
      → juniper_data.api.observability
      → juniper_data.api.__init__
      → app.py
      → juniper_data.storage  (← still initialising!)
      → ImportError: cannot import name 'LocalFSDatasetStore'

…which broke ``pytest --collect-only`` on origin/main as of
2026-05-06: 31 collection errors covering every test that imports
``juniper_data.storage`` directly or transitively. ``--collect-only``
amplifies the breakage, but a normal full pytest run also hits the
same path during fixture collection.

Fix: defer the ``set_datasets_cached`` import to inside
``CachedDatasetStore._emit_cached_count`` so the cycle never fires
at module-import time. The function is best-effort with a
``except Exception`` swallowing failures, so a deferred import that
raises ``ImportError`` (e.g. during a ``CachedDatasetStore``
instantiation in a non-API context) gets logged at DEBUG and skipped
the same way any other observability failure does.

### Verification

Full juniper-data suite under JuniperData env: **950 passed**
(was ``31 errors during collection`` on origin/main pre-fix).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pcalnon pcalnon self-assigned this May 6, 2026
Copy link
Copy Markdown
Owner Author

@pcalnon pcalnon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved

Copy link
Copy Markdown
Owner Author

@pcalnon pcalnon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved

@pcalnon pcalnon merged commit baffc42 into main May 6, 2026
25 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant